Improving Temporal Coherence of Image Features by Clustering Technique Learned from Moving Images

Object recognition is difficult because the appearance of an object changes in many different ways. To recognize objects robustly, one needs representations that are constant despite those changes. Such invariant representations can be obtained by features with low sensitivity to various visual transformations. Spatial pooling is a widely used technique for extracting invariant features from images. When the same feature is extracted from different locations of images, activations from nearby locations can be clustered together (i.e., added together) to create invariance to small spatial shifting. However, the invariance produced by spatial pooling is limited to spatial shifts. In addition, spatial pooling can only be applicable to convolutional features because spatial pooling makes clusters of features by using their spatial topography only. In this thesis, we propose a novel pooling method, auto-pooling, that learns soft clustering of features from image sequences. Auto-pooling is trained to improve the temporal coherence of features, while keeping the information loss by pooling at minimum. Our method does not use spatial information, so it can be used with non-convolutional models too. Experiments on images extracted from natural videos showed that our method can cluster similar features together. When trained by convolutional features, the auto-pooling outperformed the spatial pooling on an image classification task, even though the autopooling does not use the spatial topology of features.